Optimize DialogueFeignClient small response reader #3114
Conversation
Generate changelog in
|
|
This PR has been automatically marked as stale because it has not been touched in the last 14 days. If you'd like to keep it open, please leave a comment or add the 'long-lived' label, otherwise it'll be closed in 7 days. |
✅ Successfully generated changelog entry!What happened?Your changelog entries have been stored in the database as part of our migration to ChangelogV3. Need to regenerate?Simply interact with the changelog bot comment again to regenerate these entries. |
Avoid InputStreamReader / HeapByteBuffer overhead for small (less than 8KiB) inputs, see FasterXML/jackson-core#1081
schlosna
force-pushed
the
davids/small-feign
branch
from
e336971 to
2d19331
Compare
There was a problem hiding this comment.
Pull Request Overview
This PR optimizes the DialogueFeignClient by avoiding InputStreamReader and HeapByteBuffer overhead for small responses (less than 8KiB). The optimization reads the entire response into memory as a string for small payloads, which can be more efficient than stream-based reading for small data.
Key changes:
- Added optimization for small response bodies in the
asReader()method - Updated URL decoding to use StandardCharsets.UTF_8 directly instead of string literal
- Removed unused UnsupportedEncodingException import
| if (maybeLength != null && maybeLength < 8192) { | ||
| // Avoid InputStreamReader / HeapByteBuffer overhead for small (less than 8KiB) inputs, | ||
| // see https://github.com/FasterXML/jackson-core/pull/1081 | ||
| try (InputStream inputStream = asInputStream()) { |
There was a problem hiding this comment.
The try-with-resources block closes the InputStream after creating the StringReader, but the StringReader contains the full content and doesn't need the stream to remain open. However, this creates a potential issue if asInputStream() returns the same underlying stream instance that might be needed elsewhere. Consider ensuring this optimization only applies when it's safe to consume the entire stream.
| try (InputStream inputStream = asInputStream()) { | |
| InputStream inputStream = asInputStream(); | |
| try { |
Copilot uses AI. Check for mistakes.
There was a problem hiding this comment.
when the response is 8KiB or less we want to fully consume the response input stream and release the response after converting to an immutable String & StringReader for consumption by Jackson deserialization
There was a problem hiding this comment.
Is there a guarantee that if asReader() has been called, nothing will then call asInputStream()? I imagine it doesn't make much sense to call both, but I'm not very familiar with this API.
...a-jaxrs-client/src/main/java/com/palantir/conjure/java/client/jaxrs/DialogueFeignClient.java
Show resolved
Hide resolved
schlosna
requested review from
a team and
tpetracca
and removed request for
a team
| // Avoid InputStreamReader / HeapByteBuffer overhead for small (less than 8KiB) inputs, | ||
| // see https://github.com/FasterXML/jackson-core/pull/1081 | ||
| try (InputStream inputStream = asInputStream()) { | ||
| return new StringReader(new String(inputStream.readAllBytes(), StandardCharsets.UTF_8)); |
There was a problem hiding this comment.
Is there any way to allocate the string more directly? inputStream.readAllBytes() will allocate a byte[] and new String is going to copy that byte array into a new one, leading to twice the allocation. I'm not clear whether that is in fact more efficient than the current behavior if the length is more than 4096
| if (maybeLength != null && maybeLength < 8192) { | ||
| // Avoid InputStreamReader / HeapByteBuffer overhead for small (less than 8KiB) inputs, | ||
| // see https://github.com/FasterXML/jackson-core/pull/1081 | ||
| try (InputStream inputStream = asInputStream()) { |
There was a problem hiding this comment.
Is there a guarantee that if asReader() has been called, nothing will then call asInputStream()? I imagine it doesn't make much sense to call both, but I'm not very familiar with this API.
| // Avoid InputStreamReader / HeapByteBuffer overhead for small (less than 8KiB) inputs, | ||
| // see https://github.com/FasterXML/jackson-core/pull/1081 | ||
| try (InputStream inputStream = asInputStream()) { | ||
| return new StringReader(new String(inputStream.readAllBytes(), StandardCharsets.UTF_8)); |
There was a problem hiding this comment.
Other question: Should we guard against malicious/bad responses that return an incorrect length? (e.g. I imagine there is a corner case where the length is set to 0, which doesn't make sense)
Here we'll buffer the entire response in memory. If the content length header lied to us, we might buffer a lot more than we expected, since we just read the entire thing in memory. Should we just read the first length bytes only?
Before this PR
Small responses <= 8KiB would always allocate 8KiB
ByteBufferasInputStreamReadercreates aStreamDecoderthat allocates a fixed 8192 byteByteBuffer. This allocation becomes a scalability bottleneck for high throughput RPCs with small responses (think something returning timestamps, locks, authorization results, etc.)See https://github.com/openjdk/jdk/blob/4c03e5938df0a9cb10c2379af81163795dd3a086/src/java.base/share/classes/sun/nio/cs/StreamDecoder.java#L248
After this PR
==COMMIT_MSG==
Avoid
InputStreamReader/HeapByteBufferoverhead for small (less than 8KiB) inputs, see FasterXML/jackson-core#1081 and FasterXML/jackson-benchmarks#9 (comment) for benchmarks showing between 2x and 10x speedup handling deserialization of small values.==COMMIT_MSG==
Possible downsides?